Computational Intelligence Methods for Bioinformatics and Biostatistics

Computational Intelligence and Bioengineering %2D Essays in Memory of Antonina Starita

Computational Intelligence Methods for Bioinformatics and Biostatistics

A Novel Approach for Biclustering Gene Expression Data Using Modular Singular Value Decomposition.

A Novel Approach for Biclustering Gene Expression Data Using Modular Singular Value Decomposition.

Searching for microRNA prostate cancer target genes.

An Experimental Validation of Some Indexes of Fuzzy Clustering Similarity.
experimental validation of some indexes of fuzzy
The discriminating power of random features.


Computational intelligence and machine learning in bioinformatics.


Natural Computing Methods in Bioinformatics.
Often data analysis problems in Bioinformatics concern the fusion of multisensor outputs or the fusion of multi-source information, where one must integrate different kinds of biological data. Natural computing provides several possibilities in Bioinformatics, especially by presenting interesting nature-inspired methodologies for handling such complex problems. In this article we survey the role of natural computing in the domains of protein structure prediction, microarray data analysis and gene regulatory network generation. We utilize the learning ability of neural networks for adapting, uncertainty handling capacity of fuzzy sets and rough sets for modeling ambiguity, and the search potential of genetic algorithms for efficiently traversing large search spaces.
Natural computing methods in bioinformatics A survey.
Often data analysis problems in Bioinformatics concern the fusion of multisensor outputs or the fusion of multi-source information, where one must integrate different kinds of biological data. Natural computing provides several possibilities in Bioinformatics, especially by presenting interesting nature-inspired methodologies for handling such complex problems. In this article we survey the role of natural computing in the domains of protein structure prediction, microarray data analysis and gene regulatory network generation. We utilize the learning ability of neural networks for adapting, uncertainty handling capacity of fuzzy sets and rough sets for modeling ambiguity, and the search potential of genetic algorithms for efficiently traversing large search spaces.
Soft ranking in clustering.
Due to the diffusion of large-dimensional data sets (e.g., in DNA microarray or document organization and retrieval applications), there is a growing interest in clustering methods based on a proximity matrix. These have the advantage of being based on a data structure whose size only depends on cardinality, not dimensionality. In this paper we propose a clustering technique based on fuzzy ranks. The use of ranks helps to overcome several issues of large-dimensional data sets, whereas the fuzzy formulation is useful in encoding the information contained in the smallest entries of the proximity matrix. Comparative experiments are presented, using several standard hierarchical clustering techniques as a reference. Key words: Fuzzy rank; clustering; data mining; DNA Microarrays 1
Stability and Performances in Biclustering Algorithms.
Abstract. Stability is an important property of machine learning algorithms. Stability in clustering may be related to clustering quality or ensemble diversity, and therefore used in several ways to achieve a deeper understanding or better confidence in bioinformatic data analysis. In the specific field of fuzzy biclustering, stability can be analyzed by porting the definition of existing stability indexes to a fuzzy setting, and then adapting them to the biclustering problem. This paper presents work done in this direction, by selecting some representative stability indexes and experimentally verifying and comparing their properties. Experimental results are presented that indicate both a general agreement and some differences among the selected methods. 1
An Experimental Comparison of Kernel Clustering Methods.
Abstract. In this paper, we compare the performances of some among the most popular kernel clustering methods on several data sets. The methods are all based on central clustering and incorporate in various ways the concepts of fuzzy clustering and kernel machines. The data sets are a sample of several application domains and sizes. A thorough discussion about the techniques for validating results is also presented. Results indicate that clustering in kernel space generally outperforms standard clustering, although no method can be proven to be consistently better than the others.
Marketa Zvelebil and Jeremy Baum

A survey of kernel and spectral methods for clustering.

Applications of Fuzzy Sets Theory
Abstract. This paper presents a short survey of fuzzy and neural approaches to Information Retrieval. The goal of such approaches is to de ne exible Information Retrieval Systems able to deal with the inherent vagueness and uncertainty of the retrieval process. In this survey we address if and how some approaches met their goal. 1.
Local Learning of Tide Level Time Series using a Fuzzy Approach.

Membership Embedding Space Approach and Spectral Clustering.

Possibilistic Clustering in Feature Space.
Abstract. In this paper we propose the Possibilistic C-Means in Feature
Linear Fuzzy Clustering With Selection of Variables Using Graded Possibilistic Approach.

Vector quantization and fuzzy ranks for image reconstruction.

Fuzzy Logic and Applications
The basic concepts of fuzzy systems, particularly from the point of view of fuzzy logic based controllers, are described in this Chapter. State of the art of fuzzy control for power systems is outlined and supported by a bibliography of the literature in this area. Application of the fuzzy logic controller as a power system stabilizer is illustrated by two examples - one fairly straightforward application of fuzzy logic and one self-learning algorithm.
Possibilistic Approach to Biclustering An Application to Oligonucleotide Microarray Data Analysis.
Abstract. The important research objective of identifying genes with similar behavior with respect to different conditions has recently been tackled with biclustering techniques. In this paper we introduce a new approach to the biclustering problem using the Possibilistic Clustering paradigm. The proposed Possibilistic Biclustering algorithm finds one bicluster at a time, assigning a membership to the bicluster for each gene and for each condition. The biclustering problem, in which one would maximize the size of the bicluster and minimizing the residual, is faced as the optimization of a proper functional. We applied the algorithm to the Yeast database, obtaining fast convergence and good quality solutions. We discuss the effects of parameter tuning and the sensitivity of the method to parameter values. Comparisons with other methods from the literature are also presented. 1
Supervised Classification and Gene Selection Using Simulated Annealing.

Several Formulations for Graded Possibilistic Approach to Fuzzy Clustering.
Abstract. Fuzzy clustering is a useful tool for capturing intrinsic structure of data sets. This paper proposes several formulations for soft transition of fuzzy memberships from probabilistic partition to possibilistic one. In the proposed techniques, the free memberships are given by introducing additional penalty term used in Possibilistic c-Means. The new features of the proposed techniques are demonstrated in several numerical experiments. 1
Soft transition from probabilistic to possibilistic fuzzy clustering.
We discuss the graded possibilistic model. We review some clustering algorithms derived from the basic c-Means and introduce a formalism to provide an alternative, unified perspective on these clustering algorithms, focused on the memberships rather than on the cost function. An interesting case is the concept of graded possibility. Its formulation includes as the two extreme cases the “probabilistic ” assumption and the “possibilistic ” assumption. A possible formulation can be stated as an interval equality constraint enforcing both the normality condition and the required graded possibilistic condition. We outline a basic example of graded possibilistic clustering algorithm. The experimental demonstrations presented aim at highlighting the different properties attainable through appropriate implementation of a suitable graded possibilistic model.
Advances in fuzzy sets and rough sets.

Possibilistic clustering approach to trackless ring Pattern Recognition in RICH counters.

Shared farthest neighbor approach to clustering of high dimensionality
Clustering algorithms are routinely used in biomedical disciplines, and are a basic tool in bioinformatics. Depending on the task at hand, there are two most popular options, the central partitional techniques and the Agglomerative Hierarchical Clustering techniques and their derivatives. These methods are well studied and well established. However, both categories have some drawbacks related to data dimensionality (for partitional algorithms) and to the bottom-up structure (for hierarchical agglomerative algorithms). To overcome these limitations, motivated by the problem of gene expression analysis with DNA microarrays, we present a hierarchical clustering algorithm based on a completely different principle, which is the analysis of shared farthest neighbors. We present a framework for clustering using ranks and indexes, and introduce the Shared Farthest Neighbors clustering criterion. We illustrate the properties of the method and present experimental results on different data sets, using the strategy of evaluating data clustering by extrinsic knowledge given by class labels. 1
Unsupervised Gene Selection and Clustering Using Simulated Annealing.

Soft Rank Clustering.
Due to the diffusion of large-dimensional data sets (e.g., in DNA microarray or document organization and retrieval applications), there is a growing interest in clustering methods based on a proximity matrix. These have the advantage of being based on a data structure whose size only depends on cardinality, not dimensionality. In this paper we propose a clustering technique based on fuzzy ranks. The use of ranks helps to overcome several issues of large-dimensional data sets, whereas the fuzzy formulation is useful in encoding the information contained in the smallest entries of the proximity matrix. Comparative experiments are presented, using several standard hierarchical clustering techniques as a reference. Key words: Fuzzy rank; clustering; data mining; DNA Microarrays 1
A fuzzy approach to image analysis in HLA typing using oligonucleotide microarrays.

Integrating Conceptual Density with WordNet Domains and CALD Glosses for Noun Sense Disambiguation.

An experimental analysis of the dependence among codeword bit errors in ECOC learning machines.
One of the main factors a#ecting the e#ectiveness of Error Correcting Output Coding (ECOC) methods for classification is the dependence among the errors of the computed codeword bits. We present an extensive experimental work for evaluating the dependence among output errors of the decomposition unit in ECOC learning machines. In particular, we apply measures based on mutual information to compare the dependence of ECOC Multi-Layer Perceptron (ECOC MLP), made up by a single multi-input multi-output MLP, and ECOC ensembles made up by a set of independent and parallel dichotomizers (ECOC PND). Moreover, the experimentation analyzes the relationship between the architecture, the dependence among output errors and the performances of ECOC learning machines. The results show that the dependence among computed codeword bits is significantly smaller for ECOC PND, pointing out that ensembles of independent parallel dichotomizers are better suited for implementing ECOC classification methods. The experimental results suggest new architectures of ECOC learning machines to improve the independence among output errors and the diversity between base learners.
Effectiveness of error correcting output coding methods in ensemble and monolithic learning machines.
Error Correcting Output Coding (ECOC) methods for multiclass classification present several open problems ranging from the trade-off between their error recovering capabilities and the learnability of the induced dichotomies to the selection of proper base learners and to the design of well-separated codes for a given multiclass problem. We experimentally analyze some of the main factors affecting the effectiveness of ECOC methods. We show that the architecture of ECOC learning machines influences the accuracy of the ECOC classifier, highlighting that ensembles of parallel and independent dichotomic Multi-Layer Perceptrons are well-suited to implement ECOC methods. We quantitatively evaluate the dependence among codeword bit errors using mutual information based measures, experimentally showing that a low dependence enhances the generalization capabilities of ECOC. Moreover we show that the proper selection of the base learner and the decoding function of the reconstruction stage significantly affects the performance of the ECOC ensemble. The analysis of the relationships between the error recovering power, the accuracy of the base learners, and the dependence among codeword bits show that all these factors concur to the effectiveness of ECOC methods in a not straightforward way, very likely dependent on the distribution and complexity of the data.
Automatic Noun Sense Disambiguation.
Abstract. This paper explores a fully automatic knowledge-based method which performs the noun sense disambiguation relying only on the WordNet ontology. The basis of the method is the idea of conceptual density, that is, the correlation between the sense of a given word and its context. A new formula for calculating the conceptual density was proposed and was evaluated on the SemCor corpus. 1 An Extension of the Conceptual Density The task of Word Sense Disambiguation (WSD) consists of examining word tokens and specifying exactly which sense of each word is being used. The WordNet (WN) ontology, based on synsets (sets of synonyms), is the external lexical resource which is often used to perform the WSD task. In most of the WSD approaches, a word is disambiguated along with a portion of the text in which it is embedded, that is, its context. When the initial input source of information (i.e., the word and its context) is processed only together with the lexical knowledge source (e.g. WN), a fully automatic method which does not require any kind of training process is needed to perform WSD. Conceptual Density (CD) is a measure of the correlation among the sense of a given word and its context. The foundation of this measure is the Conceptual Distance, defined as the length of the shortest path which connects two concepts in a hierarchical semantic net. The starting point for our work was the CD formula of Agirre and Rigau [1], which compares areas of subhierarchies: CD(c; m) = Pm\Gamma 1 i=0 nhyp
Fuzzy Concepts in Vector Quantization Training.

Gene Selection Using Random Voronoi Ensembles.

An Algorithm to Model Paradigm Shifting in Fuzzy Clustering.

Application of an ensemble technique based on singular spectrum analysis to daily rainfall forecasting.

Random Voronoi ensembles for gene selection.
The paper addresses the issue of assessing the importance of input variables with respect to a given dichotomic classification problem. Both linear and non-linear cases are considered. In the linear case, the application of derivative-based saliency yields a commonly adopted ranking criterion. In the non-linear case, the method is extended by introducing a resampling technique and by clustering the obtained results for stability of the estimate.
Un M%E9todo Autom%E1tico para la Desambiguaci%F3n L%E9xica de Nombres.

Boosting and Classification of Electronic Nose Data.

Ensembles of Learning Machines.
Ensembles of learning machines constitute one of the main current directions in machine learning research, and have been applied to a wide range of real problems. Despite of the absence of an unified theory on ensembles, there are many theoretical reasons for combining multiple learners, and an empirical evidence of the effectiveness of this approach. In this paper we present a brief overview of ensemble methods, explaining the main reasons why they are able to outperform any single classifier within the ensemble, and proposing a taxonomy based on the main ways base classifiers can be generated or combined together.
Theoretical and Experimental Analysis of a Two%2DStage System for Classification.
AbstractÐWe consider a popular approach to multicategory classification tasks: a two-stage system based on a first (global) classifier with rejection followed by a (local) nearest-neighbor classifier. Patterns which are not rejected by the first classifier are classified according to its output. Rejected patterns are passed to the nearest-neighbor classifier together with the top-h ranking classes returned by the first classifier. The nearest-neighbor classifier, looking at patterns in the top-h classes, classifies the rejected pattern. An editing strategy for the nearest-neighbor reference database, controlled by the first classifier, is also considered. We analyze this system, showing that even if the first level and nearest-neighbor classifiers are not optimal in a Bayes sense, the system as a whole may be optimal. Moreover, we formally relate the response time of the system to the rejection rate of the first classifier and to the other system parameters. The error-response time trade-off is also discussed. Finally, we experimentally study two instances of the system applied to the recognition of handwritten digits. In one system, the first classifier is a fuzzy basis functions network, while in the second system it is a feed-forward neural network. Classification results as well as response times for different settings of the system parameters are reported for both systems. Index TermsÐMulticategory classification, rejection, global and local classification, hierarchical classifier, Bayes classifier. 1
NEURObjects an object%2Doriented library for neural network development.
NEURObjects is a set of C library classes for neural network development, exploiting the potentialities of object-oriented design and programming. The main goal of the library consists in supporting experimental research in neural networks and fast prototyping of inductive machine learning applications. We present NEURObjects design issues, its main functionalities, and programming examples, showing how to map neural network concepts into the design of library classes.
Possibilistic rings detection for RICH pattern recognition.
In this paper we present some results obtained using an algorithm whose core is the Possibilistic C-Spherical Shell algorithm, to detect complex images full of rings without any preliminary knowledge of number and position of the rings. This algorithm depends on the choice of several critical parameters. The statistical study described in this paper lead to high success rates even for large number of rings.
Dependence among Codeword Bits Errors in ECOC Learning Machines An Experimental Analysis.

Parallel Non Linear Dichotomizers.
We present a new learning machine model for classication problems, based on decompositions of  multiclass classication problems in sets of two-class subproblems, assigned to non linear dichotomizers  that learn their task independently of each other. The experimentation performed on classical data sets,  shows that this learning machine model achieves signicant performance improvements over MLP, and  previous classiers models based on decomposition of polychotomies into dichotomies. The theoretical  reasons of the good properties of generalization of the proposed learning machine model are explained in  the framework of the statistical learning theory.  Keywords: Generalization, learning machines for classication, decomposition of polychotomies into dichotomies, statistical learning theory.  1 Introduction  Several learning methods implementing inductive principles of empirical risk minimization [4], regularization [9], structural risk minimization [23], bayesian inference [7], mi...
Comparing decomposition methods for classification.
Decomposition methods for multiclass classication problems constitute a powerful framework to improve generalization capabilities of a large set of learning machines, including support vector machines and multi-layer perceptrons. We present a review of the main decomposition approach to classi cation and an experimental comparison of OnePer -Class (OPC), Correcting Classiers (CC) and Error Correcting Output Codes (ECOC) decomposition methods implemented using multi-layer perceptrons as dichotomizers. The results show that CC and ECOC outperform OPC over the considered data sets.  1 Introduction  Decomposition methods for classication split a multiclass problem, or polychotomy, in a series of independent twoclass problems (dichotomies) and recompose them using the outputs of dichotomizers, in order to reconstruct the original polychotomy [10, 12, 8].  Learning machines implementing decomposition methods are composed by two main units:   Decomposition Unit that analyzes the input pa...
Pattern recognition in RICH counters using the Possibilistic C%2DSpherical Shell algorithm.

Effectiveness of Error Correcting Output Codes in Multiclass Learning Problems.
In the framework of decomposition methods for multiclass classifiation problems, error correcting output codes (ECOC) can be fruitfully used as codewords for coding classes in order to enhance the generalization capability of learning machines. The effectiveness of error correcting output codes depends mainly on the independence of codeword bits and on the accuracy by which each dichotomy is learned. Separated and non-linear dichotomizers can improve the independence among computed codeword bits, thus fully exploiting the error recovering capabilities of ECOC. In the experimentation presented in this paper we compare ECOC decomposition methods implemented through monolithic multi-layer perceptrons and sets of linear and non-linear independent dichotomizers. The most effectiveness of ECOC decomposition scheme is obtained by Parallel Non-linear Dichotomizers (PND), a learning machine based on decomposition of polychotomies into dichotomies, using non linear independent dich...
Proceedings of the Third ICSC Symposia on Intelligent Industrial Automation %28IIA%2799%29 and Soft Computing %28SOCO%2799%29

NEURObjects A Set of Library Classes for Neural Networks Development.
NEURObjects is a set of C++ library classes for neural networks development, exploiting the potentialities of object-oriented design and programming. The main goal of the library is to support fast prototyping of inductive machine learning applications based on neural networks. In this paper we present the library design issues, their main functionalities, and simple examples of programming using NEURObjects.  I. Introduction  Neural networks play an important role in machine learning, in particular they permitt to efficently face problems such as regression and classification [13]. Moreover, neural networks are often relevant components of complex systems used in inductive learning tasks [12].  Nowadays, the relatively limited diffusion of neural network technology in industrial applications mainly depends on the high costs related to the long development time necessary when neural networks algorithms are implemented from scratch in order to embed those tools in new software products....
A Simulated Annealing C%2DMeans Clustering Algorithm.

A Very High Speed Industrial Neural Network Classification System Based on Custom VLSI Implementation.

NEURObjects A Set of Library Classes for Neural Networks Development.
NEURObjects is a set of C++ library classes for neural networks development, exploiting the potentialities of object-oriented design and programming. The main goal of the library is to support fast prototyping of inductive machine learning applications based on neural networks. In this paper we present the library design issues, their main functionalities, and simple examples of programming using NEURObjects.  I. Introduction  Neural networks play an important role in machine learning, in particular they permitt to efficently face problems such as regression and classification [13]. Moreover, neural networks are often relevant components of complex systems used in inductive learning tasks [12].  Nowadays, the relatively limited diffusion of neural network technology in industrial applications mainly depends on the high costs related to the long development time necessary when neural networks algorithms are implemented from scratch in order to embed those tools in new software products....
Reliability Neuro%2DControl of an Array of Sensors by Cross%2DMonitoring of Working%2DState.

A fuzzy clustering based segmentation system as support to diagnosis in medical imaging.

